Annealed f-Smoothing as a Mechanism to Speed up Neural Network Training
نویسندگان
چکیده
In this paper, we describe a method to reduce the overall number of neural network training steps, during both crossentropy and sequence training stages. This is achieved through the interpolation of frame-level CE and sequence level SMBR criteria, during the sequence training stage. This interpolation is known as f-smoothing and has previously been just used to prevent overfitting during sequence training. However, in this paper, we investigate its application to reduce the training time. We explore different interpolation strategies to reduce the overall training steps; and achieve a reduction of up to 25% with almost no degradation in word error rate (WER). Finally, we explore the generalization of f-smoothing to other tasks.
منابع مشابه
A Comparison Between Time Series, Exponential Smoothing, and Neural Network Methods To Forecast GDP of Iran
متن کامل
Numerical and Neural Network Modeling and control of an Aircraft Propeller
In this paper, parametric and numerical model of the DC motor, connected to aircraft propellers are extracted. This model is required for controlling trust and velocity of the propellers, and consequently, an aircraft. As a result, both of torque and speed of the propeller can be controlled simultaneously which increases the kinematic and kinetic performance of the aircraft. Parametric model of...
متن کاملA conjugate gradient based method for Decision Neural Network training
Decision Neural Network is a new approach for solving multi-objective decision-making problems based on artificial neural networks. Using inaccurate evaluation data, network training has improved and the number of educational data sets has decreased. The available training method is based on the gradient decent method (BP). One of its limitations is related to its convergence speed. Therefore,...
متن کاملA Differential Evolution and Spatial Distribution based Local Search for Training Fuzzy Wavelet Neural Network
Abstract Many parameter-tuning algorithms have been proposed for training Fuzzy Wavelet Neural Networks (FWNNs). Absence of appropriate structure, convergence to local optima and low speed in learning algorithms are deficiencies of FWNNs in previous studies. In this paper, a Memetic Algorithm (MA) is introduced to train FWNN for addressing aforementioned learning lacks. Differential Evolution...
متن کاملA hierarchical Convolutional Neural Network for Segmentation of Stroke Lesion in 3D Brain MRI
Introduction: Brain tumors such as glioma are among the most aggressive lesions, which result in a very short life expectancy in patients. Image segmentation is highly essential in medical image analysis with applications, particularly in clinical practices to treat brain tumors. Accurate segmentation of magnetic resonance data is crucial for diagnostic purposes, planning surgical treatments, a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2017